Caching data in memory is an efficient way to improve the performance of software. The impetus behind caching data is to increase the speed with which data may be accessed. In general, retrieving data directly from a data source may be a slow process due to transmission delays, bandwidth constraints, and computational time. The first time data is retrieved from the data source a copy of the data may be saved within a cache (e.g., buffer). Subsequent requests for the same data may be serviced using the copy of the data saved within the cache, as opposed to the original data at the data source. As such, retrieval delays for cached data may be markedly shorter when compared to retrieval delays for the original data.
In general, a cache is a buffer, smaller and faster than the original data source. Furthermore, a data consumer may be locally located to, or at least closer to, the cache than to the data source. For example, a Web Cache is a computer system in a network that keeps copies of the most-recently requested web pages in memory or on a disk in order to speed up retrieval for client computers coupled to the network.
Data caching is further performed by various operating systems (“OS”) (e.g., Windows) and virtual machines (e.g., Java Virtual Machine). Data retrieved from a hard disk is often copied into system memory and cached there by the OS while it is maintained in the general memory pool (i.e., the heap). If the heap grows beyond the confines of the physical system memory, the OS must place parts of the data in virtual memory (e.g., on the hard disk) in order to continue runtime operation. “Swapping” data to the hard disk greatly slows down processing. As such, programmers of Windows based applications should take care to expressly de-allocate memory in order to release it for other applications. Programmers of Java based applications (e.g., applets and servlets) need not expressly de-allocate objects from memory, as Java performs an automatic garbage collection without programmer intervention.
To maximize the benefits of an automatic garbage collection and/or to minimize swapping, an effective cache is desirable. In general, the larger the cache the better. However, a cache is limited by the total amount of memory less the amount of the total memory that is allocated for other uses. The amount of the total memory need for other uses usually varies with time. During intensive processing periods, other uses may consume large portions of the total memory. During less intensive processing periods, less is consumed. Since fixed sized caches must be designed for the intensive processing periods, fixed size caches do not take full advantage of the available memory at all times.